Skip to content

🐛 fix(ci): share one PR-reach reader across the three PR-visible gates (D-157) - #91

Merged
konih merged 8 commits into
mainfrom
lane/gates3-integrate
Aug 23, 2026
Merged

🐛 fix(ci): share one PR-reach reader across the three PR-visible gates (D-157)#91
konih merged 8 commits into
mainfrom
lane/gates3-integrate

Conversation

@konih

@konih konih commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

The three PR-visible text gates (hack/lint/workflow_pins_test.sh,
hack/audit/aud2_exitgate_test.sh, hack/examples/dogfood_wiring_test.sh) each
answered two questions — "does this gate actually run on a pull request?" and
"is my step wired?" — with independently-weak greps. A paths: filter at any
non-4-space indent, a - if: sequence-item disarm, and a tokenizer that split on
- and / all read as "fine".

This extracts a shared reader, hack/lib/pr_reach.sh, and cross-pins the three
gates so that deleting or disarming one is visible to the other two.

Five fail-opens closed, each verified by mutating the real verify.yaml and
running all three gates end to end:

  1. a paths: bypass that was only detected at exactly 4-space indent;
  2. a sequence-item disarm (- if: as a step's first key) grading "wired";
  3. a tokenizer splitting on - and /, so main-next yielded a main token;
  4. a CRLF fail-open;
  5. globs on branches-ignore: reading as "runs on every PR".

Scope and bounds, stated rather than claimed away. The reader is a
heuristic, not a YAML parser — deliberately: hack/audit/README.md publishes a
debian:stable-slim offline verification path, and that image has no python3,
no yq and no go. Its limits are documented in the file header and in D-157.
The final two commits are text-only: they withdraw three safety claims the code
cannot keep and disclose the remaining residuals with their measurements.

Gates on this branch: task check green (21 stages, 127 mutation controls),
task changelog-verify ok, and all three gate scripts rc=0 with the AUD2 control
floor met exactly at 57.

Decision record: D-157.

konih added 8 commits August 24, 2026 00:30
…D-157)

The three PR-visible text gates each answered "am I reachable on a pull
request?" for themselves, and two of them answered it with
`grep -qE '^[[:space:]]+pull_request:'`. A reviewer added `paths:` under
the trigger and measured rc=0: present, grep-satisfied, and disarmed for
every PR that touches only Taskfile.yml or .github/workflows/**. The
third gate never asked at all.

hack/lib/pr_reach.sh is now the single reader for all three:

  * assent_pr_reach reds on paths:/paths-ignore:, on a types: list that
    omits a GitHub default, on a branch filter excluding main, and on a
    `pull_request:` that is not a 2-space key of `on:`. It accepts the
    inline `on:` forms, the quoted "on": key, a types: superset and
    branches: [main], and refuses (fail-closed, distinct code) flow
    mappings and anchors rather than guessing.
  * assent_step_wired matches an anchored `run:` COMMAND, adds a
    job-level `needs:` check, and replaces the 1..6 line cap with the
    one-`run:`-key/no-`uses:`-key isolation invariant.
  * Each gate now cross-pins the other two's steps, so deleting any one
    of the three reds on the PR that does it.

aud2's fixed-string step check is replaced by the anchored form: with
the `run:` line commented out it reported "WITH ARGUMENTS" instead of
"does not invoke".

Every branch is mutation-controlled, including the legitimate shapes
that must stay GREEN. One fail-open was found by building the mutant:
accepting a block-scalar invocation naively let a bare `bash <script>`
line inside another step's multi-command `run: |` body pass; the block
form now counts only as a block's sole command.

CONTROL_FLOOR 38 -> 51 (text-only 32 -> 45), both met exactly.
No `check:` stage added; CHECK_STAGES untouched.
Every mutation control so far mutates verify.yaml and asks
hack/lib/pr_reach.sh about it, which cannot tell a working helper from a
stubbed one — and the helper is now a single point of failure for all
three PR-visible gates, so `assent_pr_reach() { return 0; }` would turn
them green at once. workflow_pins_test.sh and dogfood_wiring_test.sh now
also drive both helper functions against workflows written FROM SCRATCH,
whose expected answer depends on nothing under .github/workflows/**.
Verified: the stub reds them by name.

Also removes the last external anchors from the block-scalar bleed
mutants. They appended the invocation to the determinism gate's existing
`run: |` body; they now inject a self-contained `- name: unrelated` step
after `steps:`, so the control cannot red for a sibling lane's reason
when verify.yaml's toolchain steps change.
…(D-157)

Both were found by mutating the REAL .github/workflows/verify.yaml and
running the three real gate scripts against it — not by fixtures. Both
returned rc=0 before this commit; each now returns its own code.

G3-01 (P1) — the filter greps were pinned to EXACTLY four spaces
(`^    (paths|paths-ignore):`). Six-space indent is valid YAML resolving
to the identical mapping, the grep missed it, and control fell through
to `return 0`: all three gates went green on a paths-filtered trigger,
the exact shape D-157 says they refuse. It was also the one place the
helper broke its own stated rule — refusing an unevaluable SHAPE while
silently accepting an unrecognised INDENT. Matching is now
indent-agnostic within the trigger sub-block, which is safe because that
block is already scoped and exits at the next 2-space key, so a sibling
trigger's filter cannot bleed in (measured, not assumed). The sub-block's
own keys are allowlisted to GitHub's five, so an unknown key is refused
(rc=10) rather than accepted.

G3-02 (P1, inherited from main, not introduced) — the step disarm grep
lacked the `(-[[:space:]]+)?` alternative its run:/uses: siblings in the
same function had, so a step whose FIRST key is the disarm
(`      - if: …`, a sequence item) graded "wired, argument-free and
undisarmed". That hollowed out the cross-pinning this lane delivers.

G3-03 — the wiring-name-removed mutation was coupled to the literal text
of the step's `- name:` line, so an unrelated edit made the sed a no-op
and `mutate` reported a BROKEN HARNESS for a workflow that was merely
different. The step start is now derived from the gate's own run line.

G3-04 — `_assent_strip_comment`'s claim that its effect was "safe in
both directions" was false for `branches-ignore:`: stripping removes
tokens from a must-NOT-contain test, so a quoted ` #` before `main` hid
it. Stripping is now per-key and off there; the comment says why.

Why a 28-row mutant matrix missed all of this: it was broad in mutant
SHAPES and narrow in CODE coverage — 3 of 6 assent_pr_reach codes and 2
of 8 assent_step_wired codes had a fixture. Every code now has one, the
filter fixtures written at a NON-4-space indent on purpose.

CONTROL_FLOOR 51 -> 57 (text-only 45 -> 51), both met exactly.
Three fixes, all measured before and after against the two prior tips.

G3R-01 (P1) - a block sequence written FLUSH with its key, the canonical
GitHub Actions style, was misgraded, and round two's key allowlist made
one case worse. Two stacked defects, fixed separately because fixing
either alone would have looked like a close:

  * _assent_key_tokens' indent bound stopped at `RLENGTH <= ind`, so an
    item at the key's own indent was never read. Measurable at 4d592fe,
    i.e. independently of the allowlist: `branches:` + `- main` returned
    13, "excludes main", on a filter that includes it.
  * the allowlist enumerated `- main` as a sub-block KEY, so it failed
    the five-key check and returned 10 - refusing valid structure. That
    turned `branches-ignore: - gh-pages` from a correct 0 into a red.

  flush `branches: - main`            13 -> 10 -> 0
  flush `types:` superset             12 -> 10 -> 0
  flush `branches-ignore: - gh-pages`  0 -> 10 -> 0
  full legit trigger, both flush      12 -> 10 -> 0
  and the bad polarities still red:   flush `- release` -> 13,
  flush branches-ignore `- main` -> 13, flush types missing one -> 12.

G3R-02 (P1, pre-existing across both earlier commits) - branch and type
names were tokenised with `tr -c 'A-Za-z0-9_'`, which splits on `-` and
`/`. So `main-next` yielded a `main` token and a filter under which NO
pull request onto main runs the workflow reported "runs on every PR".

  branches: [main-next]        0 -> 0 -> 13
  branches: ['release/main']   0 -> 0 -> 13
  branches: seq `- main-v2`    0 -> 0 -> 13
  branches: [maintenance]     13 -> 13 -> 13  (the control that hid it:
    only values containing `main` as a SUBSTRING were broken)
  branches: [main]             0 -> 0 -> 0    (not made hostile)
  branches-ignore: [main-next] 13 -> 13 -> 0  (the false RED it also fixes)

Tokenising now splits on YAML's own separators - flow brackets, commas,
quotes, the `- ` marker, whitespace - and on nothing else. Applied to the
inline `on: [push, pull_request]` membership test too, which had the same
defect in the same direction.

G3R-03 (P2) - the `(-[[:space:]]+)?` alternative was added to the disarm
grep last round but not to the two COMMAND-matching sites, so a step
written as a bare `- run: bash <gate>` reported rc=5, "no run: COMMAND
invokes the script: deleted, commented out" - a false red with a wrong
diagnosis, for an idiom verify.yaml itself already uses three times.
Bare `- run:` 5 -> 5 -> 0, and with an argument 5 -> 5 -> 7, so the form
is graded rather than merely admitted.

Root cause of (i) and the reason it hid inside the fix for the previous
round: round two closed the CODE-coverage complaint - every reach and
wired code got a host-independent fixture - but wrote all of them at 4 or
6 spaces with sequence items indented deeper than their key, reproducing
the same indentation blind spot somewhere cheaper. Both gates that carry
the fixture set now cover flush style in both polarities.

24 before/after rows, 0 mismatches. CONTROL_FLOOR unchanged at 57/51.
…ction (D-157)

G4-01 (P1, a regression the round-3 tokenizer introduced). A single stray
CR on one `branches-ignore:` sequence item made the tokenizer emit
`main<CR>` as a whole token; `grep -qx main` missed it and the reader
returned 0, "runs on every pull request", for a trigger that EXCLUDES
main. One committed CR is enough: no .gitattributes here,
core.autocrlf=false preserves it, no gate scans workflows for CR, and a
GitHub diff renders it invisibly. CR is a YAML break character, so
`main<CR>` was never a whole token - the reader was wrong by D-157's own
"compared as WHOLE tokens" rule.

  branches-ignore: - main<CR>       13 -> 13 -> 0  -> 13   (the fail-open)
  branches: - main<CR>  (LEGIT)      0 ->  0 -> 13 ->  0   (false red)
  types: defaults, one item <CR>     0 ->  0 -> 12 ->  0   (false red)
  and every LF control unchanged.

BUT THE INSTANCE IS NOT THE POINT. Four rounds have produced six defects
of one family: a line-oriented reader inferring YAML structure from
indentation and character classes. Patching bytes one at a time was
visibly not converging, so the direction that can actually hurt is closed
structurally instead.

Every filter test here is "must CONTAIN token X" - types must carry
GitHub's defaults, branches must carry main - except ONE:
`branches-ignore:` is "must NOT contain". A misread drops or mangles a
token, so every must-CONTAIN test goes RED; the CR bug's other two
symptoms were exactly that. Only the must-NOT-contain test turns a
mangled token into a silent PASS. So the burden is inverted there: a
token carrying any byte outside the git-ref/glob set (plus `#`, since
that key alone keeps comment residue by G3-04's design) is now REFUSED
rather than searched. A future tokenizer defect on that key yields a red
or a refusal, never a silent accept - by construction, not enumeration.

A real YAML parse was evaluated and rejected on MEASURED evidence, not
assumed. python3 is an established dependency but
hack/validate-schemas-stock.sh uses `json` only, which is stdlib; PyYAML
is established nowhere here. Decisively, hack/audit/README.md publishes
`docker run ... debian:stable-slim bash hack/audit/aud2_exitgate_test.sh
--text-only` as the offline verification path, and that image measures
NO python3 and NO yq - while both that gate (51 controls) and
workflow_pins_test.sh were measured green inside it with this helper
sourced. A PyYAML reader would break a published operator path and
violate this file's no-toolchain contract (it runs before setup-go). Go
is absent from that image for the same reason.

So the reader stays a heuristic and its bound is now written into the
header rather than implied: what it reads, what it refuses, what it does
not read - and the argument that the safety property does not depend on
that list being complete.

Fixtures: CR cases at every _assent_key_tokens consumer in BOTH
polarities, built with printf so the CR is a real byte, plus the
unreadable-token refusal. 20 before/after rows across four tips, 0
mismatches. G4-06: the GATES3 backlog row now narrates rounds 3-4.
CONTROL_FLOOR unchanged at 57/51.
… (D-157)

Two P1s, both measured at GATE level on a scratch copy of the real
verify.yaml, not only at helper level - helper fixtures are what missed
them. All three gates PASSED each mutant before and red after; the
unmutated control stays rc=0 on all three.

G5-01 - a glob on `branches-ignore:` was a silent accept. `['**']`,
`['*']`, `['ma*']`, `[main*]`, `['*ain']`, flow and block form, all
returned 0 - "runs on EVERY pull request" - and every one of them
EXCLUDES main, so no PR onto main runs the workflow at all.

_assent_tokens_readable cannot catch this and must not: `*` is a legal
filter character, so the pattern tokenises perfectly, nothing is dropped
or mangled, and the burden inversion never engages. The defect was an
exact-token predicate applied to a PATTERN list. The header had already
reasoned this out one key over - a `branches:` glob reds because "a bash
gate cannot honestly evaluate GitHub's glob semantics, and refusing is
the fail-closed direction" - where the same un-evaluability is
fail-closed by polarity. On the one must-NOT-contain key it was
fail-open, so patterns are now refused there explicitly, never searched.
Literal lists still grade: [main] -> 13, [gh-pages] -> 0.

G5-02 - the disarm check matched a SPELLING, not a key. `if : `,
`"if":` and `'if' :` resolve identically to `if:` in a conformant parser
(checked against ruby/psych), so actionlint has nothing to flag - and a
step or job carrying release-exitgate's exact push-only guard graded
"wired, argument-free and undisarmed". At job level that is 0 where it
must be 4: the RELSE-08 shape this lane exists to catch.

The remedy is deliberately NOT a wider pattern. `[[:space:]]*:` would
have been the fifth spelling patch on this lane and would still miss the
sixth. Step and job keys are graded by NORMALISED NAME against an
allowlist, mirroring assent_pr_reach's filter-key allowlist, which was
measured immune to every one of these evasions. Normalisation is the
immunity. The literal greps are kept beside it as a union that can only
add detections, and a key that will not normalise to an identifier is
refused (3 at job level, 6 at step level) rather than skipped.

  step  if : / "if": / 'if' : / continue-on-error : / "c-o-e":  0 -> 8
  job   if : / "if":                                            0 -> 4
  job   needs : / "needs":                                      0 -> 9
  and the other polarity holds: a step with id:/shell:/env: in a job
  with timeout-minutes: is still 0.

G5-03/G5-04 - the guarantee sentence was an overclaim and is narrowed to
what is true. "Unknown shapes cost false reds, not silent passes" had
counterexamples INSIDE the surface documented as READ, which is the
D-124 species and worse than a documented hole. What actually holds: a
misread that DROPS or MANGLES a token fails closed everywhere (red on
the must-CONTAIN tests, refused on the must-NOT-contain one); a misread
that FABRICATES a token is not covered - `branches: ["main,next"]` is
shredded into `main` + `next` and grades 0. Measured, recorded in
DOES-NOT-READ, and left unfixed this round on purpose: the claim is
corrected to match the code rather than the code trusted to match the
claim.

25 helper-level before/after rows and 15 gate-level ones, 0 mismatches
in the after column. CONTROL_FLOOR unchanged at 57/51.
…r (D-157)

Text only. Zero non-comment lines changed in hack/lib/pr_reach.sh -
verified by diffing out comment and blank lines. No predicate touched,
no mechanism added, the filter path and _assent_tokens_globfree
untouched. This is the parking edit, not a seventh round of fixes.

G6-01. Reproduced before rewording: `_assent_step_keys` derives the step
key indent with `match($0, /^[ ]*- /)`, which matches exactly one space,
so `      -   if : ${{ false }}` yields 8 while the keys sit at 10. The
enumeration visits nothing, `?unreadable` never fires, and
assent_step_wired returns 0 - WIRED - for a step carrying
release-exitgate's own push-only guard. Measured at this tip AND at
01b2322, so the shape predates the normalisation work and survives it;
with a BARE `if:` under the same wide marker it returns 8, because the
literal grep still reaches it. Two sentences were therefore false and
are withdrawn:

  * "their union can only add detections, and they still catch a key at
    an indent the enumeration does not visit" - the union covers bare
    spellings at any indent and non-bare spellings at indents the
    enumeration visits; the intersection is covered by neither and is
    SKIPPED.
  * "anything normalisation cannot reduce to an identifier is refused
    rather than skipped" - that holds for what the enumeration visits
    and says nothing about what it does not.

The shape is added to DOES NOT READ with its measurements and a
one-command reproduction.

G6-02. "DOES NOT READ: multi-line flow sequences (refused)" was wrong
twice - they are read, and the six shapes measured here
(branches:/types:/branches-ignore:, both polarities) all graded
correctly. Moved to READS, with no correctness guarantee claimed beyond
those six.

A third falsifiable claim, found while re-checking this row against
G6-01 rather than reported to me: "a future tokenizer defect on that key
can produce a red or a refusal, never a silent accept - by
construction". False for a defect that drops the token set ENTIRELY. An
empty set passes _assent_tokens_readable, passes
_assent_tokens_globfree and passes `grep -qx main`, so the consumer
falls through to 0 - measured on all three. Same shape as G6-01 on the
other layer: an enumeration that visits nothing yields an empty set, and
an empty set satisfies every must-NOT-contain check. No total-drop
defect is known on that key today; what is withdrawn is the claim that
the construction rules one out. The must-CONTAIN tests are unaffected -
an empty set cannot contain the required token, so they red (measured:
empty -> 13).

Also scoped the READS list to the filter reader, since the step reader's
bounds differ and stating them together read as unconditional.

Behaviour unchanged and re-verified: all three gates green, the round-4
and round-5 before/after matrices re-run at 0 mismatches, bash 3.2 green.
R7-01. The same sentence was withdrawn in D-157 and in the header, and
left standing a third time inside _assent_tokens_readable's own doc
comment - the function whose burden inversion it exists to justify, and
the one a reader lands on without seeing the header 140 lines up. It now
says a defect that MANGLES a token yields a red or a refusal, and states
plainly that a defect dropping the token set ENTIRELY is not covered,
with a pointer to the header bullet carrying the measurements.

Free while here, neither blocking:

R7-02 - the comment claimed assent_pr_reach "normalises and ALLOWLISTS"
the filter keys. It allowlists and refuses non-matches; it does not
normalise. The immunity claim is true and measured (paths : / "paths": /
'paths' : / "types" : all refuse at 10), only the mechanism description
was loose. Both sites grade a structure rather than a spelling; the
mechanisms differ and the comment now says so.

R7-03 - "Every function returns a distinct numeric code and prints
NOTHING" was literally false: five internal helpers print to stdout by
design, which is how they compose. Narrowed to the two grading entry
points, with the property that actually matters stated separately - no
function here emits a finding or writes to stderr (verified: zero
occurrences of >&2 in the file).

Executable surface byte-identical to 8e76336: comment- and blank-
stripped renderings have the same md5.

Also adds backlog row GATES3-F01 for a drift found by the same review
and deliberately NOT fixed here - hack/audit/README.md:214 publishes the
AUD2 control floor as "38 full, 32 text-only" while the code says 57/51.
That file is outside this lane's fence, and quietly correcting a
published number is the same species as the drift.
@konih
konih merged commit 8c2d919 into main Aug 23, 2026
7 checks passed
@konih
konih deleted the lane/gates3-integrate branch August 23, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant